
- 1 - 

^CE DETECTION IN DIGITAL HvSgES 

Technical Field of the Invention 

The present invention relates to digital colour images and, in particular, to the 
detection of faces in colour digital images. 

^ Background Art 

Colour digital images are increasingly being stored in multi-media databases, 
and utilised in various computer applications, m many such applications it is desirable to 
be able to detect the location of a face in a visual image as one step in a multi-step 
process. The multi-step process can include content-based image retoeval, personal 
10 identification or verification for use with automatic teller maclunes or security cameras, 
or automated interaction between humans and computational devices. 

Various prior art face detection methods a^ known including eigenfaces, neural 
networks, clustering, feature identification and skin colour techniques. Each of these 
techniques has its sti-engths and weaknesses, however, one feature which they have in 
15 common is that they are computationally intensive and therefore very slow, ox they are 
fast but not sufficiently robust to detect faces. 

The eigenface or eigenvector method is particularly suitable for face recognition 
and there is some tolerance for lighting variation, however it does not cope with different 
viewpoints of faces and does not handle occlusion of various facial features (such as 
20 occurs if a person is wearing sunglasses). Also it is not scale invariant. 

The neural network approach utihses traimng based on a large number of face 
images and uon-face ^ages and has the advantages of bemg relatively shnple to 
implement, providing some tolerance to the occlusion of facial features and some 
tolerance to lighting variation. It is also relatively easy to improve the detection rate by 
25 re-training the neural network using false detections. However, it is not scale invariant. 
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does «ot cape with ftent viewpoints or orientation, and leS^o an exhaustive process 
to locate faces on an image. 

The clustering technique is somewhat similar to the eigenface approach. A pixel 
window (eg. 20 x 20) is typically moved over the image and the distance between the 
5 resulting test pattern and a prototype face image and a prototype non-face image is 
represented by a vector. The vector captures the similarity aad differences between the 
test pattern and the face model. A neural network can then be trained to classify as to 
whether the vector represents a face or a non-face. While this method is robust, it does 
not cope with different scales, different viewpoints or orientations. It leads to an 
10 exhaustive approach to locate faces and relies upon assumed parameters. 

The feature identification method is based upon searching for potential facial 
features or groups of facial features such as eyebrows, eyes, nose and mouth. The 
detection process involves identifying facial features and grouping these fean^es into 
feature pairs, partial face groups, or face candidates. This process is advantageous in that 
15 it is relatively scale invariant, there .s no exhaustive searching, it is able to handle the 
occlusion of some facial features and it is also able to handle different viewpoints and 
orientations. Its main disadvantages are that there are potentially many false detections 
and that its perfonnance is very dependent upon the facial feature detection algorithms 
used. 

20 The use of skin colour to detect human faces is described in a paper by Yang J 

and Waibel A (1995) "Tracking Human Faces in Real-Time" CMU^CS-95-210, School of 
Computer Science, Carnegie Mellon University. This proposal was based on the concept 
that the human visual system adapts to different levels of brightness and to different 
illumination sources which implies that the human peroeption of colour is consistent 

25 Within a wide range of environmental lighting conditions, It was therefore thought 
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possible to remov#ghtness from the skin colour repx^tation while preserving 
accurate, but low dimensional, colour infonnation. As a consequence, in this prior art 
technique, the chromatic colour space was used. Chromatic colours (eg. r and g) can be 
derived from the RGB values as; 

^ '• = R/(R + G + B) and g = G/(R + G + B) 

These chromatic colours are known as "pur«" colours in the absence of 

brightness. 

Utilising this colour space. Yang and Waibel found the distribution of skin 
colour of different people, including both different persons and different races, was 
10 clustered together. Tins means «.at the skin colours of different people are very close and 
that the main differences are in differences of intensity. 

This prior art method first of all generated a skin colour distribution model using 
a set of example face images from which skin colour regions were manually selected, 
•men the test image was converted to the chromatic colour space. Next each image in the 
15 test image (as converted) was then compared to the distribution of the skin colour model. 
Finally, all skin colour pixels so detected were identified, and regions of adjacent skm 
colour pixels could then be considered potential face candidates. 

This prior art method has the advantage that processing colour is much faster 
than processing individual facial features, that colour is substantially orientation invariant 
20 and that it is insensitive to the occlusions of some facial features. The system is also 
substantially viewpoint invariant and scale invariant. However, the method suffers from a 
number of disadvantages includmg that the colour representation of the face can be 
i^^^^^^^^y^^E-I^^iins conditions, and that different cameras (eg. digital or film) 
can pn,duce different colour values even for the sa^ne person in the same enviromncnt. 
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However W^nificant disadvantage of the prior J^ethods is that the skm 
colour model is not very discrixninating (ie. selecting pixels on a basis of whether they are 
included in the skin colour distribution results in a lot of non-skin colour pixels being 
included erroneously). It is also difficult to locate clusters or regions of skin colour pixels 
5 that can be considered as candidate faces. 

Disclosure of the Invention 
An object of the present invention is to provide an improved method of detecting 
one or more faces in digital colour images. 

In accordance witt, a flr,, aspect of the present invention there i. disctosed a 
10 method of detecting a face in a colo,^ d,gita. in,age fonned of a plurality of pixel., said 
rnethod comprising the steps of; 

(i) testing the colour of said pixel, to delenmne those said pixels having 
pr^lo^inanfly skin colour, said tes.i.>g utiUsing at least one in«ge capture condiHon 

provided with said image; and 

15 (ii) subjecting only said those pixels detennined m step (i) as having 

predominantly skin colour to further facial feature analysis whereby those said pixels not 
having a predominantly skin colour are not subjected to said further facial feature 

analysis. 

Preferably each image_c^e_co„dWonJs at a time_th^image is 

) caeSTCd. Advantageously, thejmagejsengg^cording to a predetenninedjonna. 
and at least on. image capture condition is represented as meta-data associated with the 
fonnat. Most P-fe.bly the « ieaa.,nO!nage^^ 
conditions at a time the image was captured. 

m a par^cular in^ementation~step (i), comprises the sub-step, preceding said 

testing, of: 
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said image into a plurality of regions^ch said region comprising 
a plurality of said pixels; and 

wherein said testing is perfbimed on pixels within each said region to detennine 
those ones of said regions that are predominantly skin colour and step (ii) comprises 
5 performing said further facial feature analysis on only those said regions determined to be 
predomitiantly of skin colour. 

m accordance with another aspect of the present invention, there is disclosed a 
method of detecting a face in a colour digxtal image, said method comprising the steps of 

(i) segmenting said image into a plurality of regions each having a 
10 substantially homogenous colour; 

(ii) testing the colour of each said region created in step (i) to 
deteimine those regions having predominantly skin colour; and 

(iii) subjecting only the regions detemiined in step (ii) to further facial 
feature analysis whereby said regions created in step (i) „ot having a predominantly skin 

15 colour are not subjected to said further feature analysis. 

Apparatus and computer readable media for performing the invention are also 
disclosed 

Brief Description of the Drawings 
A number of embodiments of the present invention will now be described with 
20 reference to the drawings in which: 

Fig. 1 is a schematic representation of the pixels of a colour digital image; 
Fig. 2 shows the segmenting of the image of Fig. 1 into a plurality of regions 
each having a substantially homogenous colour according to a first embodiment; 

Fig. 35 is a flow chart of a face detection process according to the first 

25 embodiment; 
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Fig. 4 is Aematic block diagram of a general p^^se computer upon which 
embodiments of the present invention can be practised; 

Fig. 5 is a flow chart depicting the generation of a face colour distribution model; 
Fig. 6 is a flow chart of a fece detection process according to a second 
5 embodiment; and 

Fig. 7 is a flow chart of a face detection process according to a third 
embodiment. 

DetaUed Description including Best Mode 

Fig. 1 illustrates a typical colour digital image I having a size of 832 x 624 
10 pixels 5, each of which has an RGB value. 

According to a first embodiment of the present invention, rather than consider 
the Skin colour of the image on a pixel by pixel basis as described above in relation to the 
Prior art of Yang and Waibel. the image 1 is segmented into a number of regions. An 
example of such segmentation is schematically illustrated in Fig. 2 on the segmentation 
15 basis that all the pixels in each region 2 have substantially the same colour. Alternatively 
the image may be segmented into arbitrary regions for processing. 

The first embodiment implements a process 30 illustrated in the flow chart of 
Fig. 3, in which the regional segmentation of the image is carried out at step 3 1 . Next, the 
regions of the image are converted in step 32 into the chromatic colour space (as 
10 described above). The next step 33 is to select those of the regions determined in step 3, 
which have a specified percentage (typically in the range 90-95o/„) of pixels having a skin 
colour. These selected regions are then conveniently represented by a boundary box or 
other boundary indication. Finally, at step 34 the selected regions, including any 
combinations o^ overlapping regions, are subjected to a fluther analysis (preferably not 
S based on skin colour) to determine if the selected region(s) represent a face or faces. 
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This initi Aour grouping can use any region basft,lour image segmentation 
technique. Preferably the image is partitioned into colour regions by seeking connected 
groups of pixels which have sixnilar colours over a local region. Very small isolated 
initial spatial groupings can be ignored in order to find major colour regions and reduce a 
5 noise effect. A representative colour of each initial spatial region is detennined by an 
average colour value of the region. 

A colour region starts fi-om an arbitrarily chosen pixel, which is compared with 
its neighbouring pixels. The region size is increased by adding neighbouring pixels, 
which are similar in colour, using a colour similarity threshold T A neighbouring pixel is 
10 added to the region if |Rp-Rm|<T and |Gp-Gm|<r and [Bp-Bm|<T, where Rp. Gp, Bp are 
R. G, B values of the neighbouring pixel and Rm, Gm, Bm represented average R, G. B 
values of the region. 

When a region has no more neighbouring pixels of similar colour, the region 
stops growing and represents one of the initial spatial groupings. If the region size ,s 
15 below a predetermined threshold value, it is ignored. A region having a pixel number 
equal or greater than the predetermined threshold is represented by its average colour. 

A new pixel which does not yet belong to any region is chosen to start a new 
colour region. The process continues until each pixel in the image either belongs to an 
initial spatial grouping or has been ignored as being part of a small region. 
20 The initial spatial groupings provide a colour region segmentation of the image 

witii each region being represented by its average colour. 

m tins way. for most images where the bulk of the zmage is not a face, or part of 
a face, tixe majority of pixels will be grouped into regions or objects (be they foreground 
or background, fete) which are clearly not faces. Therefore these non-facial objects can be 
?5 quickly eliminated on the basis of their colour. 
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Once tb. have ac^^ u>ey a« S^.cru^ into ft. "pu^e" 

ehronnattc c<..o„r .pace u^n.in, U,c e,na«on. given above » as .o provide . and g valne. 
A generou. mle auch aa a ml. a,at a, leas, 85% of ft. pi„,a wiftin a given „gion be of 
&ee colour can be usd « aelec, ftose region. „orfty of toher examination. Prefembly, 
5 *e .eat for face colour .alces into account the nature of the orig.naI in,age, for example 
whether d,e image was talcen wim or without a flash. This information can be determined 
flom the image source, eg. a camera. 

Thereafter, only ftose selected regions are subjected to a itather test to detennine 
the presence of facial fea«^. Tliis tether test provides a conclusive derermirration as to 
- whedrer or not a region conatinites a face. In this connection, the iteher tes, is Ukely to 
be computationally slower and therefore the above described elimination of regions 
ensures a,at tire computationally slow meftod is only appKed to relatively small portions 
ofthcoveraU image. Thus fte total processing time .s reduced. Acc„,^,«ly. 
meftod performs a computa^onally simple process on most, if not all pixels, and then 
onlyperlbims complex examination on skin colour regions. 

The pref^red meftod of verifying if a region represents a face rehes upon edge 
detection techniques as a means o, detecting facia, features. In particular feial features 
such as eyes, eyebrows and mouths often appear as dark bar, on a face and thus provide 
dark edges. 

The preferred form of edge detection is use of an edge detection filter. This 
utilises two fimctions operating in orftogona. directions. To detect a horizontal bar a 
second derivative Gaussian luncHon is used in fte vertical direction and a Gaussian 
function is used in the horizontal direction. 

Once an idge has be«. determined in this way each detected edge is examined 
Any pair of detected edges can be found to be derived from, and «,„s be indicative of. a 
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pa-r Of eye., a pair^yeb^w.. an ey. ^ a«ooia.cd eft>„, ^ ^ 

»laave position and «ze o, U,. deeccted edges. Stanariy. aa individual edge can be 
derived .ion, and be indica«v. of. a n.„„d, if i. i, ,ooa.ed a. an a^„prfa.e posi.on 
relative to the eyes and/or eyebrows alieady detected. 
5 By proceeding in this fashion, a given region begins to accumulate facial features 

building from skin tone though eyebrows/eyes and then to a ntouth. Tie n.„re taci^ 
fean.es found for a given „gion which .s a face candidate, the greats the possibility that 
the candidate actually is a face. 

Furthermore, flte above described method has the advantage that i, is able to 
10 cater for the citoumstance where a face is backgrounded against a background .egion of 
substantiauy the same colour. Under these ciroutnstances, in Yang and Waibel. method 
no boundao, be^.een the face and the baclcground would be hMy to be detected 
n-erefore the region aa a whole would be selected for tunhcr testing. However, the 

15 decisions about which pixels are skin colour. Conse^ently the face is mo„ likely to be 

separated from the background Ih addition th« ^^.u a ■ 

^ a. m addition, the method is naturally independent of 

orientation or partial occlusion of the face. 

Ftohennote. dte above method also permits «se positives to be examined at dte 
««her stage and the^ore does not exclude from subse<,uent testing regions which are 
20 Itkely to be ultimately detennined as a facial region. 

The to. embodiment described above notes that the nature of the ori^al image 
may be taken into account when performing an initial face detection process. P„r«,er 
embodiments to be now described build upon this feamre. 

When an image is captured using a camera it is necessaty either for the petson 
25 taking the picture to manually establish the camera settings (such as shutter speed. 
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aperture, focal len^Rtc), or for the camera to perfonn ^operation automatically. 
Whichever is the case, the settings of the camera directly effect the appearance and 
quality of the image taken. In particular, the perceived brightness, colour, and sharpness 
of the objects within an image all depend on how the settings of the camera are 
configured. For example, it is possible to take two pictures of the same scene with 
different camera settings and to obtain two images in which the same objects appear with 
different colours and brightness. Therefore, the ability to calibrate (in particular) colour 
infomiation contained in (digital) images enables a broad variety of object detection and 
classification tasks in which colour is a strong discriminating feature. 

Face detection is one such example apphcation, and the present inventors have 
detexmined that the creation of face colour distribution models (CDM's), each adapted to 
specific lighting conditions, that can improve both the accuracy and reliability of face 
detection. Variations in lighting conditions can result firom the use of a flash, such being 
a feature recognised as contributing in the face detection method of the first embodiment. 
Since lightness is representative of colour features such as luminance and chrominance, 
such features may be used to quantify face detection. 

Before an image can be processed using a face colour distribution model, the 
face colour distribution model must be constructed. This is perfonned according to a 
method 50 shown in Fig. 5. The method 50 firstly gathers image samples at step 52 that 
are representative images that contain faces, the images being acquired under a variety of 
lighting conditions and thus indicative of changes in luminance and chrominance. These 
images ar« then manually examined in step 54 to extract regions of skin colour for further 
processing in model formation. Step 54 may be performed by manually drawing a 
bounding box around a sample of face coloured pixels. Step 56, which follows, derives 
colour representation values for the extracted pixels. This may be perfonned by 
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«u>rforming tbc Mui p«eb into a percepmal colour .uch as CE L'u-v or 

CIE L.a-b, «, fl« each pixe, is represented by „ leas, a 2-du„e„siooal vector. 
Alternatively other colour spaces such a. HX5 .nd HSV may be used. Preferably each 
pixel is represented as a leug,h-3 vector incon-orating both the l™run,„ce and 

5 chrominance values. 

The colour representation values of pixels are then divided at step 58 into a 
nun^ber of sets (58a. 5Sb ... 58„) accordntg to the lighting conditions present when each 
of the image, wee capntred. Example sets are flash, non-flash, indoor, outdoor, and 
combinations of these. Altematively. lighting parameters obtained dit^tly fiom the 
10 camera such as the operation of a flash, may be used to identify and distinguish the sets. 
Other Bghting conditions such as bright or cloudy, dusk or dawn, or a type of artificial 
light such as fluorescent, incandescent or halogen, may be used or detected for these 
purposes. These details may be p,ov, ded by means of human input a. the time of image 
capture. 

'5 For each of the s«s (58a ... 58„) of face samples, step 60 then constructs a 

corresponding colour disn^buUon model (CDM) (60a ... 60n) that best fits tire samples of 
fi.ce colour pixels. Ihe CDM can be a histogram, a p™bability density Wtion. or a 
binao^map. ^ ™bo*ncn, a mixmre of Gaussian PDF-, are « to the sample data 
using techni^es toown in the ar, such as the expectation maximisation (EM) algorithm 
20 with cither cross-validation. .acklCfe, and bootstrap techmques being used to estimate 
the goodness of fit of the model . 

When each CDM (60a ... 60n) has been constructed, it is then desirable as 
shown in step 62 to establish a corresponding probabUity threshold (62. ... 62n) below 
Which a colour vettor is to be Cassifled as relating to a non-face pixel, and above which 
25 the colour vector is to be Cassifled as a potential face pixel. AdditionaHy. the face colour 
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pr»b.bi.,y can A U, ftcia. un^. s^. ,e,aiW below "in 

fte prcfc^ed «=bo<,u„«.,, fte CDM is co„s«,c,ed »„„, cclou. r^rcs^r^^ v^u.. 
derived u^ing a percephu, colour space (such aa Cffi L-u-v or CIE L-a-b) and .hen 
Tanafonned back into 0,e colour fonna, of fte input image, ie., cifter RGB or YUV. Thia 
5 removea the neceaaity for tranafon^ng the input image into the perceptual colour .pace. 

Since different unage capmre devices have differing perfom,a„ce. often 
determined by the quaUty and size of optical components (eg. lens, mirrors, apertur. etc ) 
.>picaUy a CDM or a set of CDM's are gen««ed for a particular capmrc device. In one 
implementation, where the image capt^e device (eg. camera) includes a hgh, meter a 
10 reading ftom the Ught meter at ti.e mom^tt the image was capUired can be used to 
■.etcm,ine the required CDM. I„ this ftshion. a greater range of colour models may be 
devised and can be selected witirout possible human i„,erfere„„. ^uch interfer^ice may 
occur where the human user manuaUy selects the oper^ion of the flash where otherwise 
automatic operation o, the flash would not be required. Further, the flash/outdoors 
IS example sets above ^ve rise to four (4, sets of CDM-s. Using a light meter with, say 4- 
bit encoding, can pmvidc si«een (.6) models. Also, use of a li^t meter provides for 
enhanced r^ducability of results and ^tables the face samples used to gene^te the 
models to be taicen under labomtory conditions and installation at the time of camera 



manufacture. 

20 



The processing 70 of an image according to a second embodiment is shown in 
Fig. 6. An input image is pmvided at step 72 and a, step 74 U>e hghting conditions under 
wh.ch the image was captured are detennined. Such a detetmtnation may be based on 
bmary data obtained directiy ftom the camera (eg. flash*indoors. no iiash^outdoot, 
no.flash^dooH. flash^utdoors) or co^esponding meta-data provided with » 
25 accompanying, the hnage. which may be encoded or otherwise communicated accotdmg 
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or clos^ CDM « soleoted a bank of look-up .ab.es 78 retaining «,e CDM'. (60a 
60n, p^vioualy defined. A, .«p 80. a ft. p„e, of «.e inpu. in,ag= 72 i. .elected and 
a. step 82 is tested ,„ sec if the (rob or YUV) colour of pi^el is contained in the 

5 seJected CDM (60a . . . 60n). 

The steps sl,„wn in Fig. 6 following flte contpaHson of step 82 depend upon d.e 
manner in which the CDM's are stored^ ^ a preferred in,p,e„>en.a,ion, the selected 
thresholds of step 62 (Pig 5) arc used to constnact a binary ntap or look-up table, where a 
.epresentaHve colour vector is represented by a . if it is a colour vector that is contained 
10 Within the thresholdcd ftce colour disMbution, and by a 0 if the eolcur vector does not 
occur in the thresholded colour distribution. Alten^tively, the CDM ntay represent the 
ti-iuencies of the repres^tative colour vecto. of the thresholded colour disttibution (ie 
the CDM is effectively a histogram of representative colour vectors). A further vadation 
.s the case where a sa„,p,ed distribnUon is appro^naated by a par«ne.ric model such as a 

IS Gaussian or a mixture of Gaussians. I„ the latter case th. rrav. 

lauer case, me CDM compnses the 

parameters of the model (eg. mean, co-vaiiance) 

As seen m Fig. 6. and according to the prefetred implem«ttation. a 1 or 0 value 
arising irom step 82 is added to a map in step 84. Step 86 d«enni„es if there are more 
Ptxels in the image to be processed and step 88 obtains and passes the next pixel to 
20 step 82 for appropHate testing. When all pixels have been tested against the selected 
CDM, step 90 indicates the resuh of the preceding steps as being a binary t^e image map 
formed using detected skm-ooloured pixels. 

The map is dren subj^ted at step 92 to finther analysis of the skin-colo„r«i 
Pixels to provide a* step 94 a ftce de^tion map for the image. The ft«her anal>.is „f 
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colour. 



In practice, binao- 6c. »,ap f„„ned a. ...p 90 n,ay c<,n«i„ ar<«s where eifter 
are ama., oon-face pixel. (0-,) s^o.^., face pixela C •a), or vice ve„. One 
5 approach for ftr,hcr analysis according ,o step 92 is processin, h^ary face i„,age 
map so as „ sc. » 0 any pixel locations which are contained in areas ^. are smaller «.an 
the smallest size of a pot^tial face and ,o se, any 0 pixel locations .o . if ,hey a„ 
»— ed by lively colo. pixels. Ms n«y he performed using a pair of 

morpholo^cal opening and Cosing operations wift suiti.hly shaped s^cn^ring elenre„.s. 
0 A first structuring element such as: 



0 110 
1111 
1111 
0 1-10 



■s used in the openin, operation ,o remove potential ^ candidate pixel locations below 
this size. A second structuring element such as: 

[ii]- 

is used in the closiug operation to f.,1 any holes in potential face candidate pixel locations 
Alteniativ. approaches to the use of tiie stmcturing elements include using a 
Hough ^onn, or to count the number of pixels in the tegion having sldn colour and to 
threshold «,a. count against a predete^ined petcenugo value. Other ntethods n.y be 

used to perform these tasks. 

^e result of the process 70 of Rg. 6 is a face detection map of pixel locations in 
«.e .nput iniage a. which a face has been detected, and in aU likelihood, a face is present. 
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The afo™»on«i edge detecSon of £U«h.r processing U.e Itolj, ftce 

P«e.. .o d.«„^„e if a face exi« n^y ..e. he perfonncd on U.e ^e de.cc.on n,3p 94 
resulting ftom the method 70. 

In a,c prcf«red embodiment, the fice colour distribution models are built for a 
5 number Of separate lighting conditions, such as flash, non-flash, indoor, outdoor, and the 
lice. However, this technique may be extended to the more ge„e^ ease of arbitrary 
hghting conditions based directly on parameters obtained fc>m the camera. A lis. of 
camera parameters that may be of use in ftis situation is as follows: 
(i) white balance; 
° (ii) white balance mode 

(iii) aperture (iris); 

(iv) shutter speed; 

(v) auto gain control (AGC); 

(vi) auto exposure (AE) mode; 

(vii) gamma; 

(viii) pedestal level; and 
O'x) flare compensation. 

Tie parametets obtained Irom the camera are preferably obtained torn a mc.a- 
da« s«am associated wi«. .he capture of each image (or video sequence). Examples of 
such nansmission protocols include IEEE .394 (-arewlre"). AUo the ISO standards have 
deflned methods for «taching meta-data to image, and video in MPEO-T. MPEG4 
and JPEG. 

Wh„st the first en^bodinaent described with reference to Figs. 1 to 3 divides the 
.mage according to regions of substantially homogeneous colour, the second «„bodi.ent 
and a third embodiment, are not so constrained. 
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n.e tedWodtaen, is depicted in Fig. 7 by alfo-od .50 »here a, inpu, 
.-ge 152 is provi..a „^ „ 

» steps 74. 76 and 7S. .espec«v.„, „f second e»bodin,en.. Once U.C app^p J 
CDM lu. been selected in step ,56. step 160 fo,W to p„„ss the inpu. i„ag. as one or 
S n.o„ region,. ^ , „^„^ ^^^^^ ^^^^ ^ ^ 

P..el baris. Altemaavely. the inpu, i^age may be geometdcaUy divided into sintple pixe. 
Blocks (eg. 25 X 25 pixels. ,0 X 20 pixels) wbich can be Wed and ptocessed in raster 
order. A further alternative, is where the regions. Mice the first embodinren. are divtded 
on die basis of substantially homogeneous colour. 

Step ,62 selects a firs, region to be processed and step 154 a first pixel of that 
tcg-on. Step ,66 compares the selected pixe, with the CDM in a manner cotresponding to 

step 82 of the second embodiment. Where Hie nixel ,„..-i, ... 

■vncrc me pixel matches the model, step 168 

mcrcments a count of pixels in ^ region meeting «iat criteria. Step ,70 determines 
whemer «tcre are any other pixels is the region to be processed and . if so. step 172 
15 Obtains the next pixe, and renmis to step ,66 for appropriate testing. When a,, p„e,s in 
die region have been processed, step ,74 follows to compare the peioenrag, of pixels 
Classified for die region as skin colour against a predetermined percentage d«sho,d 
value. Where die percenUge is less .Han the predetennined mmiber. die ..gion is 
considered a non.f.ce region and srep , 76 follows m .est if there arc any more regions to 
20 be piocessed. If so. step ,78 select, die next region and returns processing to step 164 
The count is then re-set If no^ the method 150 ends a, step ,84. 

Where the percentage exceeds die predetennined pen^entage, the region is 
considered a possible f^e region and step ,80 follows to assess the region according to 
fisher facia, detection a„a,y^s. Where such anSy^ia ,oes not detect a face die 
2S mediod 150 proceeds to step ,76 m process any furrier regions. Where die fiinher 
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returns to step 176. 

An example of «,e ^ analysis able ,c be perfonnec. a= a co^cquenee of 
appropriate ma*,., a, .«p ,so Is fl.e edge ^^^^ ^ ^^^^^^ 

5 to the first embodiment. 

The above described embodto«,B each indicate that face detection in in,ages 
may be perfomted as a two stage pn^ess. a ^^enting something akin to a &s, 
fiUering of the image to obtain ,ike,y candidate pixels o, regions, and the second 
representing more d.orough analysis to provide an actual determination on those pixels or 

10 regions passed by the first staee In each r*ac^ n^t,^-- 

ini. stage, in each case, hghting conditions associated with the 

capmre of the image contribute to the detemiination perfcimed by the first stage. 

The above described methods.are preferably practiced nsing a conventional 
general-puipose computer system 100, such a, that shown m Fig. 4 where ttie piocesses 
Of Fig. 3 and/or Figs. 5 and 6 are implemented as software, such as an application 
15 P"^ executing withh. the compter system 100. In particular, the steps of the 
mea^ods are effected by mstructions in the softwan: that are carried out by the computer. 
The software may be divided mto two separate parts; one part for carrying out *e above 
method steps; and another par, to manage the user interface between the Utter and the 
user. The software may be stored fa a computer readable medium,, including the storage 
20 devices described below, f,r example. Tt. software is loaded into die computer ftom the 
computer readable medium, and then executed by «ie compute. A computer readable 
medium having such software or computer program recoiled on it is a computer program 
Ptoduct. TTie us. of the computer p„gram product m the computer prefe^bly effects an 
advantageous appbtams for detecting face candidate „gions in accordance with the 
25 embodiments of the invention. 
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The cor^My^^ 100 cc„,pHses a con.pu,er „cft ,01, i„p„. acvic« .ch 
a. a .=,*oaM ,0. ,03. »u^„. .evicc i„o,«.i„, . ^„ , ^^^^ 

device 1 14. A Mo*Uau..D«.c.„:,„, (Mode.) .^.^ H« is u.e. 

computer module 1 01 for communicating to and from » 

umcann^ to and from a commumcations network 120, for 

5 example connectable via a teleohone li^^Tot 

a telephone hue 121 or other functional medium. The 

modem 116 can be used to obtain access tr, th^ i^* 

Dtam access to the Intemet, and other network systems, such 

as a Local Area Network (LAN) or a Wide Area Network (WAWs u • 

c<* iMerwork (WAN), these being possible 

sources of input images and destinations for detected faces. 

The computer module 101 typically includes at J^=.cf 

yi^iv^any mcluaes at least one processor unit 105 a 

10 n,^ory ^.,06. for e.a«p,e fo^ad ™d„..o, ^don. a.,=e.. n,e.o, 

(RAM) and read only „cmc^ (roM). ,„p„,,ou.p« (^o) i«erf-a=es toc.udin^ a video 
interface ,0.. and an .O in^ace „3 .or *e .e.Boar. ,02 and n„„se ,03 and 
opuonauy a JoysUc. (no, iUua,ra,=d). and an interface ,08 for „oden, „6. A „„,age 
<lev.ce ,09 i. p^vided and .„ica„y inCudes a hard di^ drive „0 and a floppy di* 
dnve,,,, A magneto «pe drive (no. i„„„ra.ed) naay a,ao be used. A CD-ROM 

dnv. ,12 i, typically provided as a non-volatile source of data Th. 

B suuicc oi aata. The components 105 

to 113 of the computer module 101 ivnircU-u ^ 

e loi. typically commumcate via an interconnected bus 104 

and in a manner which results in a conventional mnH. 

oonventional mode of operation of the computer 

.ysten, .00 ..own to d,ose in the relevant art. H.a»p,es of compute, on which the 
enrhodiments can he practised include BM-PC's and co^patihles. Sun Sparcstationa or 
alike computer systems evolved therefrom. 

T«. the appucauon progran, of the prefetred e^hodi^ent is residen, „. 
the hard dia. d.ve „0 and read and contt^Ue. in its execution hy the processor ,05 
intermediate storage of «.e p,o^ ^ ^ 

.S acconrp^shed usin, the senUconductor ntentory ,0.. possih.y in concen with the hard 
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encoded o„ a CD-ROM or floppy di». and read via a,e corresponding drive or , H 
or a..e™Uve,3, ,e read .he user fron. ..e r,e.wc* 1.0 via the n,oden, device „«' 
Sm fUrtt-er. «.e aoftware can a,so be leaded i..o «,e con,p„»r .y«eM ,00 iron. o1.er 
5 co„p„.er readable ™edi„™ i^Cndin, „,agne«c .ape, a ROM or i„.egra,ed ci.c* a 
™.op«cal disl. a radio or i„^..ed .ran^is.io„ channel be«.een tt.e con,pu,er 
module 10. and another device, a computer readable can! auch a. a PCMCIA card, and 
.he h,ten,e. and Intranets including e-mail transmiaaions and information recoMed on 
Websites and are bice. Tie fbregoing is metely exemplary of relevant compter reliable 
10 mediums. Other computer readable modi™. „ay be practiced wia>ou. departing ftcn, 
the scope and spirit of the invention. 

The further processing to candidate face in^ages and regions may also be 
perfonned by or using the cotnputer system 100 and Ia.own a^angements for such 
processing, 

IS TT,e m«hod of detecting face candidate regions may alte„,advely be 

m^plemented in dedicated hardware such as one „r more ,ntegra.ed circuit perfonning 
.he factions or sub fintctions of Fig. 3 and/or Figs. 5 and 6. Such dedicated hardware 
may include g^phic processors, dig-tal signal processors, or one or more nncrop„cesso« 



and associated memories. 

20 



Industrial Applicability 

It is apparent from the above that the embodiments of the invention are 
applicable in fields such as content-based image retrieval, personal identification or 
verincation for use with automatic teller machines or security cameras, or automated 
interaction betweei humans and computational devices. 



CFPI327USA IPR20A 4«15e4USDl 



[Ir\ELEaClSRA\IPR\Ipr20a]46I584USCl.doc;ldp 



spmt of the invention as defined in the claims. 



4 ; ; 
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